[META 388] Proposal for collecting Azure App Service cloud metadata #365

russcam · 2020-11-11T04:03:56Z

This commit updates the cloud metadata spec to propose collecting Azure App Service metadata.

Azure App Services are a very popular PaaS offering on Azure. In contrast to Azure VMs, App Services do not have access to the internal metadata endpoint in order to retrieve metadata about the app instance. Instead, much of the metadata that is relevant to the purposes of APM are available in environment variables.

The environment variables of interest are documented here.

This commit updates the cloud metadata spec to propose collecting Azure App Service metadata. Azure App Services are a very popular PaaS offering on Azure. In contrast to Azure VMs, App Services do not have access to the internal metadata endpoint in order to retrieve metadata about the app instance. Instead, much of the metadata that is relevant to the purposes of APM are available in environment variables. The environment variables of interest are documented here.

apmmachine · 2020-11-11T04:06:46Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Build Cause: Pull request #365 updated
Start Time: 2021-03-09T02:45:30.589+0000
Duration: 5 min 54 sec
Commit: dd3b003

Trends 🧪

mikker

LGTM.

How do we determine what PaaS-platforms to include checks for? I feel a little bit like as soon as we start adding some, we can never stop adding new ones as they pop up 😅

mikker · 2020-11-11T08:01:43Z

We could let it be up to the individual agents what platforms to support, so the PaaS' are optional.

Azure's is probably way more used for .NET than for Ruby, for example, where Heroku might be the exact opposite situation.

axw

Looks good! What method should be used to identify that you're running within Azure App Service? Presence of valid WEBSITE_* environment variables?

We could let it be up to the individual agents what platforms to support, so the PaaS' are optional.

++

graphaelli · 2020-11-11T19:54:37Z

specs/agents/metadata.md

+| `instance.id`         | `WEBSITE_INSTANCE_ID` |
+| `instance.name`       | `WEBSITE_SITE_NAME` |
+| `project.name`        | `WEBSITE_RESOURCE_GROUP` |
+| `provider`            | azure               |


Is there value in capturing the cloud product, perhaps cloud.product: azure-app-service as well? It appears we'd need a new cloud ecs field.

++ I think there could be. cloud fields look to be very much intended for VMs e.g. availability_zone, machine_type.

With elastic/ecs#1204 we will have an ECS field for this as of next release - cloud.service.name. elastic/apm-server#4625 is tracking addition to the intake for this.

+1 for somehow capturing the fact that this set of metadata is being captured in an azure-app-services environment in order to make it distinct from data collected via the instance endpoints. I was thinking similar thoughts when I investigated this for node.js recently specifically around a theoretical problem of a user looking at two sets of azure data and wondering why different fields are collected from each.

specs/agents/metadata.md

trentm · 2020-11-16T22:31:45Z

specs/agents/metadata.md

+##### Azure App Services
+
+Azure App Services are a PaaS offering within Azure which doe not
+have access to the internal metadata endpoint. Metadata about


What @axw said in an earlier comment:

What method should be used to identify that you're running within Azure App Service? Presence of valid WEBSITE_* environment variables?

The presence of WEBSITE_* environment variables and the format of WEBSITE_OWNER_NAME should be sufficient.

Co-authored-by: Trent Mick <[email protected]>

russcam · 2020-11-23T05:12:04Z

We could let it be up to the individual agents what platforms to support, so the PaaS' are optional.

I think this is a reasonable approach. Perhaps I should mark Azure App Services as optional in the spec?

…nto azure-app-service-metadata

russcam · 2020-11-25T05:47:27Z

Marked Azure App Service Metadata as optional.

I think the only thing left is to determine a timeline for implementation. @alex-fedotyev / @graphaelli, thoughts on where this might slot in? Happy to write a prototype implementation for the .NET agent too.

russcam · 2020-12-08T07:35:24Z

I've opened elastic/apm-agent-dotnet#1083 as an example implementation

graphaelli · 2020-12-09T18:44:03Z

@alex-fedotyev and I discussed this. He's going to investigate adoption across agents and help set priority on this.

alex-fedotyev · 2020-12-10T08:28:52Z

@russcam - did you check if our current logic for collecting cloud metadata would not work for Azure App Services?
I am wondering whether it is already detecting cloud provider and region + availability zone details.

Also instance ID would be interesting to leverage in setting up service.node.name, how does the agent assign the node names today for AppServices?

Did you check if WEBSITE_ env variables are present in the AppServices for Linux? I remember leveraging them on Windows, but not sure about Linux.

Azure Functions also set WEBSITE_ environment variables, technically they are part of the App Service infrastructure.
But functions also set additional env var FUNCTIONS_EXTENSION_VERSION. Would we want to check that?

From my experience, I recall .NET/Java/Node.js and PHP being well adopted on Azure AppService platform. This looks to me as an improvement worth of doing across at least those language agents.

astorm · 2021-01-25T18:41:09Z

specs/agents/metadata.md

+| `project.name`        | `WEBSITE_RESOURCE_GROUP` |
+| `provider`            | azure               |
+| `region`              | last part of `WEBSITE_OWNER_NAME`, split by `-`, trim end `"webspace"`          |
+


@russcam datapoint: When I deployed a node application via azure app services my region looked like

"WEBSITE_RESOURCE_GROUP": "appsvc_linux_centralus",

i.e. the WEBSITE_OWNER_NAME was delimited with underscores characters (_) and not the hyphen/dash. You can see this here (will be slow in coming up if the app's gone idle)

What do you think about spec-ing this as splitting by - OR _ -- or perhaps even "splitting by non-alpha-numeric-characters"?

Thanks for the additional data point. Looks like there may be some cross-language and cross-platform differences in environment variable values. For example, the node app value is

"WEBSITE_OWNER_NAME": "7657426d-c4c3-44ac-88a2-3b2cd59e6dba+appsvc_linux_centralus-CentralUSwebspace-Linux"

The extracted region should be CentralUS, though the value includes a -Linux suffix which wasn't present on the .NET app service.

I think what might be good for me to do before updating the PR is to deploy some other language app services to see if there is more variation in values. It might be a simple case of updating the WEBSITE_OWNER_NAME rule to substring from 0 up to start of webspace, then split by - and take the last value.

specs/agents/metadata.md

Co-authored-by: eyalkoren <[email protected]>

russcam · 2021-02-19T05:29:41Z

I've added some gherkin specs to indicate what success and failure scenarios 9768039. With these specs, agents can generate the steps for a test

basepi · 2021-03-08T17:52:23Z

What's the status on this spec? Theoretically it's slated for 7.12 which is in two weeks. Can we wrap it up and get it merged so agents can implement? Alternatively we could push it to 7.13, but we should make a decision on that.

/cc @AlexanderWert

This commit updates the Azure App Service Metadata spec to indicate that webspace(.*) should be trimmed from the last part of `WEBSITE_OWNER_NAME` split by '-', in order to get the region.

russcam · 2021-03-09T02:49:44Z

@basepi good question :)

I believe with the addition of gherkin specs with scenarios to demonstrate expected outcomes, this is good to merge- I added a link to the specs and clarified how region is obtained, based on @astorm's input in #365 (comment). I deployed empty Azure App Services for node and python and couldn't replicate the WEBSITE_OWNER_NAME format seen, but it's a small logical change to accommodate in the spec.

basepi · 2021-03-09T15:55:13Z

In that case, let's make this the last call for input on this spec. @russcam is going to merge it in two days (March 11) unless there is more feedback.

russcam · 2021-03-11T12:43:20Z

Thanks for everyone's input, merging this in.

russcam requested review from a team as code owners November 11, 2020 04:03

mikker approved these changes Nov 11, 2020

View reviewed changes

axw reviewed Nov 11, 2020

View reviewed changes

SergeyKleyman approved these changes Nov 11, 2020

View reviewed changes

graphaelli reviewed Nov 11, 2020

View reviewed changes

trentm reviewed Nov 16, 2020

View reviewed changes

basepi approved these changes Nov 18, 2020

View reviewed changes

Update specs/agents/metadata.md

e262bba

Co-authored-by: Trent Mick <[email protected]>

vigneshshanmugam approved these changes Nov 23, 2020

View reviewed changes

russcam added 2 commits November 25, 2020 15:42

Add optional note for Azure App Service cloud metadata

2259cd4

Merge branch 'azure-app-service-metadata' of github.com:russcam/apm i…

e737a81

…nto azure-app-service-metadata

AlexanderWert added the meta label Dec 9, 2020

AlexanderWert added the apm-agents label Jan 21, 2021

AlexanderWert added this to the 7.12 milestone Jan 21, 2021

astorm mentioned this pull request Jan 25, 2021

feat: add azure app service env var mappings #406

Closed

astorm reviewed Jan 25, 2021

View reviewed changes

specs/agents/metadata.md Show resolved Hide resolved

AlexanderWert added the agent-dotnet label Feb 1, 2021

russcam and others added 3 commits February 19, 2021 11:36

Update specs/agents/metadata.md

d459a6e

Co-authored-by: eyalkoren <[email protected]>

Merge branch 'master' into azure-app-service-metadata

af4b6ea

Add gherkin specs for Azure App Metadata

9768039

russcam and others added 2 commits March 9, 2021 12:22

Merge branch 'master' into azure-app-service-metadata

c86dce2

Update spec and link to gherkin specs

dd3b003

This commit updates the Azure App Service Metadata spec to indicate that webspace(.*) should be trimmed from the last part of `WEBSITE_OWNER_NAME` split by '-', in order to get the region.

russcam merged commit 8b4578b into elastic:master Mar 11, 2021

russcam deleted the azure-app-service-metadata branch March 11, 2021 12:45

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[META 388] Proposal for collecting Azure App Service cloud metadata #365

[META 388] Proposal for collecting Azure App Service cloud metadata #365

russcam commented Nov 11, 2020

apmmachine commented Nov 11, 2020 •

edited

Loading

Build stats

Trends 🧪

mikker left a comment

mikker commented Nov 11, 2020 •

edited

Loading

axw left a comment

graphaelli Nov 11, 2020

russcam Nov 23, 2020

graphaelli Jan 14, 2021

astorm Jan 25, 2021

trentm Nov 16, 2020

russcam Nov 23, 2020

russcam commented Nov 23, 2020

russcam commented Nov 25, 2020

russcam commented Dec 8, 2020

graphaelli commented Dec 9, 2020

alex-fedotyev commented Dec 10, 2020

astorm Jan 25, 2021

russcam Jan 27, 2021

russcam commented Feb 19, 2021 •

edited

Loading

basepi commented Mar 8, 2021 •

edited

Loading

russcam commented Mar 9, 2021

basepi commented Mar 9, 2021

russcam commented Mar 11, 2021

[META 388] Proposal for collecting Azure App Service cloud metadata #365

[META 388] Proposal for collecting Azure App Service cloud metadata #365

Conversation

russcam commented Nov 11, 2020

apmmachine commented Nov 11, 2020 • edited Loading

💚 Build Succeeded

Build stats

Trends 🧪

mikker left a comment

Choose a reason for hiding this comment

mikker commented Nov 11, 2020 • edited Loading

axw left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

russcam commented Nov 23, 2020

russcam commented Nov 25, 2020

russcam commented Dec 8, 2020

graphaelli commented Dec 9, 2020

alex-fedotyev commented Dec 10, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

russcam commented Feb 19, 2021 • edited Loading

basepi commented Mar 8, 2021 • edited Loading

russcam commented Mar 9, 2021

basepi commented Mar 9, 2021

russcam commented Mar 11, 2021

apmmachine commented Nov 11, 2020 •

edited

Loading

mikker commented Nov 11, 2020 •

edited

Loading

russcam commented Feb 19, 2021 •

edited

Loading

basepi commented Mar 8, 2021 •

edited

Loading